The development of the EDVAC computer system of 1948 is often cited as the beginning of the computer era. Since that time, computer systems have evolved into extremely sophisticated devices that may be found in many different settings. Computer systems typically include a combination of hardware (e.g., semiconductors, integrated circuits, circuit boards, etc.) and software (e.g., computer programs).
The typical design methodology for integrated circuit designs—such as very large scale integrated (VLSI) circuits and application specific integrated circuits (ASICs)—is conventionally divided into the following three stages. First, a design capture step is performed using, for example, a high-level language synthesis package. Next, design verification is made on the resulting design, which includes simulations, timing analysis, and automatic test pattern generation (ATPG) tools. Finally, there is layout and eventual tape out of the device. The device is then tested, and the process may need to be reiterated one or more times until the desired design criteria are satisfied.
The design capture step typically involves the specification of a logic circuit by a designer. A hardware description language (“HDL”) provides the designer with a mechanism for describing the operation of the desired logic circuit in a technology-independent manner. Automated software tools take an HDL description of the integrated circuit (sometimes referred to as a behavioral or register-transfer-level description) and map it into an equivalent netlist composed of the standard cells from a selected standard cell library. This process is commonly known as “synthesis.” Other software tools take a netlist comprised of standard cells and create a physical layout of the chip by placing the cells relative to each other to minimize timing delays or wire lengths, and then create electrical connections (or routing) between the cells to physically complete the desired circuit. Once a netlist has been generated from the logic design, silicon compilers, also called place and route tools, convert the netlist into a semiconductor circuit layout. The semiconductor circuit layout specifies the physical implementation of the circuit in silicon or other semiconductor materials.
Design verification involves verifying that the logic definition is correct, that the circuit implements the function expected by the designers, and that the many optimizations and transformations introduced during the design process do not alter the intended logical function of the design. Design verification may involve timing analysis and simulation tools to assess the functionality of the design. If errors are found or the resulting functionality is unacceptable, the designer modifies the design as needed. These design iterations help to ensure that the design satisfies its requirements. Formal verification (property checking) may also be used to prove correct behavior for selected aspects of the design. Formal verification is a technique that models a logic circuit as a state transition system using specifications for components in the system.
Another verification method is to generate large numbers of testcases or test programs. A traditional strategy is to manually write directed testcases that exercise specified parts of the design and run those testcases on a simulator that models the device operation. Writing such testcases is very time consuming and lacks random testing, which means covering all the legal or valid combinations of stimuli may be difficult. Also, the testcases written in this way are difficult to maintain because they lack portability and are vulnerable to any and all design changes, which causes significant modifications to the testcase bucket or regression test suite for almost every design change.
Another approach to creating testcases is to use a random testcase generator, which is often a script written in a high level language that requires the verification engineer to provide some input parameters as to what type of testcase is required. The testcase generator than randomly generates testcases based on the input parameters, which may include the commands, legal or valid ranges, and/or values for various parameters that need to be varied/changed in order to test the design. The input parameters may be passed to the testcase generation engine as a file (e.g., a parameter file). The valid commands, parameter ranges, and/or values are defined based on the design specification and the directedness of the testcases required. This approach, besides providing comparatively increased random testing, also reduces the time required to generate testcase buckets or regression test suites because the job of the verification engineer is now merely to provide the constraints to the testcase generator after understanding the design specifications and verification environment. Unfortunately, merely generating testcases randomly lacks accountability, so the testcases may run for hours with little awareness of whether they are generating, encountering, or hitting the full range of output signals or events needed to validate the design under test.
In an attempt to address the problem of lack of accountability, another technique for determining the completeness of the verification effort is to compile a comprehensive list of output signals or events that represent the various scenarios needed to validate the design under test. The problem with this technique lies in generating the proper input parameters and testcases to hit each of these events. A technique for generating the proper testcases is to harvest only testcases that hit the desired events as specified in a parameter file. The drawback to this harvesting technique is that it depends on a static bucket of testcases, or a semi-automatic generation of testcases, but with static inputs to the generation process. Thus, this harvesting technique does not guarantee that a current set of running testcases will contain a test that actually hits the desired events, and this harvesting technique may consume a significant number of simulation cycles and hardware and software resources to hit the necessary events.
Thus, without a better way to generate testcases, testing of devices will continue to suffer with a lack of complete coverage.